Near Optimality of Quantized Policies in Stochastic Control Under Weak Continuity Conditions

نویسندگان

  • Naci Saldi
  • Serdar Yüksel
  • Tamás Linder
چکیده

This paper studies the approximation of optimal control policies by quantized (discretized) policies for a very general class of Markov decision processes (MDPs). The problem is motivated by applications in networked control systems, computational methods for MDPs, and learning algorithms for MDPs. We consider the finite-action approximation of stationary policies for a discrete-time Markov decision process with discounted and average costs under a weak continuity assumption on the transition probability, which is a significant relaxation of conditions required in earlier literature. The discretization is constructive, and quantized policies are shown to approximate optimal deterministic stationary policies with arbitrary precision. The results are applied to the fully observed reduction of a partially observed Markov decision process, where weak continuity is a much more reasonable assumption than more stringent conditions such as strong continuity or continuity in total variation.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Robustness to incorrect priors in partially observed stochastic control

We study the continuity properties of optimal solutions to stochastic control problems with respect to initial probability measures and applications of these to the robustness of optimal control policies applied to systems with incomplete or incorrect priors. It is shown that for single and multi-stage optimal cost problems, continuity and robustness cannot be established under weak convergence...

متن کامل

Control Theory and Economic Policy Optimization: The Origin, Achievements and the Fading Optimism from a Historical Standpoint

Economists were interested in economic stabilization policies as early as the 1930’s but the formal applications of stability theory from the classical control theory to economic analysis appeared in the early 1950’s when a number of control engineers actively collaborated with economists on economic stability and feedback mechanisms. The theory of optimal control resulting from the contributio...

متن کامل

On the optimality equation for average cost Markov decision processes and its validity for inventory control

As is well known, average-cost optimality inequalities imply the existence of stationary optimal policies for Markov decision processes with average costs per unit time, and these inequalities hold under broad natural conditions. This paper provides sufficient conditions for the validity of the average-cost optimality equation for an infinite state problem with weakly continuous transition prob...

متن کامل

Convex Analysis in Decentralized Stochastic Control, Strategic Measures, and Optimal Solutions

This paper is concerned with the properties of the sets of strategic measures induced by admissible team policies in decentralized stochastic control and the convexity properties in dynamic team problems. To facilitate a convex analytical approach, strategic measures for team problems are introduced. Properties such as convexity, and compactness and Borel measurability under weak convergence to...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2015